FURTHER ADVENTURES WITH CORRECT-IT: Z80DOS, PUBLIC FILES AND SETDRU
                             - or -
                 'WHERE'D THAT FILE COME FROM?'

by Rick Charnes
San Francisco, Oct. 18, 1987


    Hooray - I've cracked the code!
     
    ...

    Ever notice how sometimes the most valuable lessons in life 
come from errors you've made?

                         * * *
  
    For the last several days I've been been using fellow Morrow 
owner and programmer par excellence Carson Wilson's new operating 
system, Z80DOS.  This new BDOS for Z80 computers provides full 
datestamping (and time-stamping, if a real time clock is present) 
integrated in to the directory.  ZRDOS, the BDOS used with the 
"official" Z-System has no internal datestamping, so it is 
therefore generally done via an external program, DateStamper.  
DateStamper, however, is fully compatible with Z80DOS and one can 
quite readily have both date/time-stamping environments 
simultaneously operative and this is indeed how I have arranged 
my system.  I have been running the two side by side with no 
problem for several days.
         
    (I will henceforth refer to "date and time-stamping, if a 
real time clock is present" as simply "datestamping.)
         
    The original CP/M BDOS of course has no datestamping.  Since 
historically CP/M programmers had no OS support the utiltiies 
they developed almost without exception were without 
datestamping.  Although there are now several other CP/M 
replacements that do have datestamping -- QP/M, DOS+, SUPERDOS 
and P2DOS noteworthy among them -- we have had a situation where 
there was scarcely anything in software we could do with their 
datestamps save for running the small number of utilities 
supplied with them!  In addition, a fairly sophisticated 
programming protocol was necessary in order to incorporate 
datestamping into the various CP/M utilities and applications.  
There certainly has been no standard.  This has indirectly been 
the weakness of all previously existing date/time-stamping CP/M 
BDOS replacements.  This is of course in marked contrast to the 
MS-DOS world where to my knowledge all programs are integrated 
with a datestamping system built into the OS.
         
    Z80DOS, however, does its datestamping by implementing two 
new BDOS functions exclusive to Z80DOS which allow programs that 
modify or copy files to maintain file date and time stamps with 
very little programming overhead.  It does this in a way that 
none of the above BDOS'es or even DateStamper can do.  This means 
that programmers can include datestamping support in new progams, 
or -- significantly, add it to existing programs -- with very 
little difficulty or extra code.  Because of this ease of 
programming we now have great potential for creating a standard.
         
    One major advantage of this for me has been the existence of 
a utility supplied with Z80DOS -- a counterpart for which exists 
for none of the other systems -- that uses these new functions to 
enable a user to do something that I have missed sorely: the 
retention over successive editings of the 'date of creation' 
stamp of a word-processed file.  (In all systems the 'date of 
last modification' stamp is of course renewed and changed 
accordingly.)  Because of the way CP/M word processors modify 
files, on all other CP/M-compatible datestamping systems when a 
word-processed file is edited and modified the date of creation 
stamp suddenly becomes the same as the date of modification!  
This new feature, by the way, in my opinion makes Z80DOS' 
datestamping system superior to that of MS-DOS which does not 
have a function for 'date of creation' at all but rather only for 
last update.
   
    I commend Z80DOS to all CP/M users and urge all interested 
parties to investigate its superlative features.  It is in the 
public domain and may be found on BBS's as Z80DOS10.LBR.

                         * * *

    But this isn't why I'm excited.

    One of the other features of Z80DOS is its implementation of 
a sytem whereby certain files, usually overlays and the like, can 
be made available from all user areas in a drive.  These files 
are then considered 'public'.  Some provision of this sort is 
implemented in most of the CP/M 2.2 enhancements, including CP/M 
3.0 and ZRDOS.  In different systems it is implemented in 
different ways.  With ZRDOS one has 'public directories' wherein 
any file residing inside that directory is automatically made 
public.  With Z80DOS one sets the file attribute bit of the 
second character of the filename.  For all intents and purposes 
the net result is the same.  
         
    I have been using the public directory feature of ZRDOS for 
the year and a quarter I've been using ZCPR3.  One of the 
programs for which on a hard disk it is quite literally 
indispensable is the Morrow spell-checker Correct- It.  Without 
some sort of public file feature one can only run Correct-It from 
a single user area -- unless, that is, one has a copy of the 96k 
DICT.BIN, the 2k DINDEX.BIN and the 24k FIXUP.COM on each and 
every user area on one's hard disk!
  
    As implemented in most BDOS's including Echelon's ZRDOS, 
however, there is one danger to something as powerful as this 
'public' feature.  Erasing, or even simply creating, writing to 
or copying a non-public file with the same name as a public file 
erases the public file!  Needless to say this can be quite 
disconcerting and if you don't know what's going on trying to 
trace the cause of your misfortune can be extremely frustrating.
         
    Z80DOS comes equipped with a safety feature to deal with this 
problem.  A public file can only be erased if it is in the 
currently logged user area.  Otherwise a read-only error is 
considered to have occurred.

    But I didn't know all this at the time.

    I was sitting pretty correcting a file one day, my mind as 
far away as could be from public files, safety features and the 
like -- and I got hit with my first R/O error message.  I 
couldn't figure out what was happening.  I was running my CORRECT- 
IT alias that I had always used with ZRDOS; what could be 
different this time?  And to make things worse, when the error 
message displayed on the sceen it mentioned something about a 
file called 'XXPED.$$$'.  What the heck was that and what was 
going on?
         
    It took me a couple of times running the alias before I 
realized that something that was happening was probably related 
to this public file safety feature.  I then recalled how ZRDOS 
worked with this alias (previously released in my CORREC10.LBR).  
One must have AUXDICT.TXT, DICT.BIN, DINDEX.BIN and FIXUP.COM all 
in the public directory area, thus making them public files.  But 
it does something else: when FIXUP.COM writes your newly 
"learned" words to your old AUXDICT.TXT, one eventually notices 
to one's distress that it ends up putting the resultant file not 
back in the public directory area where it must be located at 
alias invocation but rather in the currently logged user area!
                  
    I have since figured out that what happens is similar to how 
all CP/M word processor work when they modify ("write to") a 
presently-existing file.  A new file is created with a temporary 
name, the old one is erased, then the new is renamed to the old.  
(This, by the way, is why datestamping systems have such 
difficulty maintaining the time/date stamp of a word-processed 
file: they're essentially working with a new file.  Once the 
original file is erased the new file has no way of knowing what 
the old file's creation date was.)
         
    So ZRDOS' public feature finds AUXDICT.TXT since it's in a 
public directory, opens it and reads its contents, and proceeds 
to create a new file with a temporary name.  Since we're probably 
not logged on the public directory this new file is written to 
the currently logged user area.  When that is done FIXUP.COM 
looks again to AUXDICT.TXT, still a public file, and erases it.  
Finally the temporary file residing on the currently logged user 
area is renamed to AUXDICT.TXT and no one is any the wiser.

    This worked fine with ZRDOS.  The only drawback is that at 
alias exit we were then left with an AUXDICT.TXT that was no 
longer in a public directory, not a public file and therefore 
inaccessible to the next run of CORRECT-IT.  So I dealt with this 
problem easily by putting one more command at the end of the 
alias that has the utility MOVE.COM move it from where it was 
back to the public directory.  No actual copying takes place; 
only the user number is changed in the directory and the 
operation takes place extremely quickly.
         
    So why wasn't this working with Z80DOS?  It took quite a bit 
of head-scratching to figure this out, especially since my 
understanding of what was happening with ZRDOS as above only came 
to me after going through the process of figuring out the error 
with Z80DOS.  I hadn't the slightest idea what 'XXPED.$$$' was.  
All I knew is that I was always finding a file by this name left 
on the currently logged user area, and it was exact replica of my 
original AUXDICT.TXT only with the new learned words added.  This 
was the file that AUXDICT.TXT _should_ now have been.  I still 
hadn't yet pinned down exactly where the error was occurring.  
All I knew is that ZEX file that my alias invoked was aborting 
somewhere right in the middle and I was getting a R/O error.
                  
    So I did what I've taught myself, actually with Ted 
Silveira's help, to do in these situations:  simplify.  Take 
everything line by line, one by one, operation by operation and 
study exactly what is going on with each.  Strip out all 
unnecessary overhead.  I ran the alias several times with 
printouts in front of me of both the ZEX batch file and the alias 
and paid careful attention to what was going on on the screen.  I 
realized that it was happening immediately after the disk 
finished its grinding from having written the new AUXDICT.TXT to 
disk.  I looked and thought, thought and looked, and studied some 
more, and finally realized: something somewhere is trying to 
erase AUXDICT.TXT and the operating system isn't letting it.  It 
was then that I pieced together the scheme above of how 
the new AUXDICT.TXT gets written.  It was of course the Z80DOS 
safety feature:  FIXUP.COM was trying to erase AUXDICT.TXT, a 
public file, and the OS wouldn't let it.  
                  
    But I was still puzzled by this mysterious 'XXPED.$$$'.  I 
assumed it was some sort of temporary file, but where was it 
coming from?  The BDOS?  I looked through the code to the Z80DOS 
source, which is provided along with Z80DOS10.LBR.  I found the 
routines for file deletion and R/O errors, hoping to find that it 
creates a file named (for some reason) XXPED.$$$ when it is asked 
to erase a public file and the anti-deletiion safety feature is 
invoked.  I did a string search and found -- nothing.
             
    I then hit upon another idea.  I loaded up ZPATCH, the ZCPR3 
file patcher that makes the CP/M ZAP look like a 1979 version of 
ED.COM, and took a look at the binary code inside FIXUP.COM.  
Eureka!! -- there it was:  XXPED.$$$.  I'm quite certain only God 
and Correct-It's author know where the name 'XXPED.$$$' is 
supposed to mean, but it's quite obviously the name FIXUP.COM 
gives to its temporary file just before it erases the old 
AUXDICT.TXT.  I guess the author couldn't think of a good name 
for it and gave it the name that he felt was the most 
XXPEDient...

                         * * *

But what is the "code" that I say I cracked?  I'm getting to that.

    So what was I to do now?  The BDOS, perhaps quite properly, 
will not let me erase the old AUXDICT.TXT.  How is FIXUP going to 
work properly?
         
    I suppose I could add several new commands to the alias:

(1)  Log on to the specific user area on which the old 
     AUXDICT.TXT is located

(2)  Erase it.

(3)  Make the new XXPED.$$$ a public file.

(4)  Rename it to AUXDICT.TXT
         
    How boring.
    
    Enter SETDRU.
    
    SETDRU is one of those good old classic CP/M utilities that 
few people use anymore but were as valuable as a gold mine in 
their heyday.  It does, as a standalone program, what an 
operating system's public feature does generally.  It is used to 
modify other COM files.  A program modified by SETDRU puts a 
"filter" in high memory which then redirects all the program's 
internal "calls" made to other, supplemental and subsidiary files 
it may need for its operation -- overlays, other COM files, text 
files, etc -- to specific, user-specified user numbers.  It is 
perfect for our purposes.

    But I had tried and failed, six or nine months ago, to get 
SETDRU to work properly on Correct-It.  I had initially felt that 
the extra step at the end of the ZRDOS alias that MOVEs 
AUXDICT.TXT back to the public user area was inelegant and 
unprofessional.  Computers shouldn't have to work like that.
         
    In order to get SETDRU to modify a file properly you must 
provide it the name of each and every subsidiary file to which 
the main program makes a call, and then specify the user number 
to which you wish to redirect said call.  This is often difficult 
to do.  A call can be either a "read" or a "write", and both must 
be specified.  I remember nine months agao sitting and spending 
two or three hours with Correct-It, tracing every operation and 
guessing at when DINDEX.BIN is being invoked, when DICT.BIN is 
being read, the exact order of invocation, etc.  After some time 
I managed to get the whole shebang working successfully --- until 
the time that FIXUP should have written the new learned words to 
AUXDICT.TXT.  
         
    I just couldn't figure out how to get that right.  After 
prompting me for "Name of auxiliary dictionary to write to?  " 
I would enter 'AUXDICT.TXT' and the operation would abort; it 
acted as if it were unable to find the file, even though I knew 
it was there.  I knew that some call was being made, but I was 
damned if I could find out what, who, when, where or how.
         
    I gave up.  I wrote the simple MOVE command line into the 
alias and forgot about it.  
    
    Until last night, nine months later.

    I figured: it must be XXPED.$$$.  That was the secret.  It 
was the missing call.  I sat down, found my yellowed, dog-eared 
notes from my SETDRU experience of those many moons ago, put my 
programmer's cap back on and loaded up SETDRU one more time.  I 
went through all the same call redirections I found on my paper, 
but this time additionally redirected FIXUP to XXPED.$$$, and 
specified the user number.  Exited SETDRU, turned off the public 
file feature on all relevant files, called up my alias, and held 
my breath.
         
    It worked like a charm.  CORRECT.COM found FIXUP, FIXUP found 
DICT.BIN, and FIXUP finally did "find" AUXDICT.TXT.  Hard work, 
study -- and learning from your errors -- does pay off.  Thank 
you, my new friend, XXPED.$$$.

                             * * *
             
    Here's how to do it:  put CORRECT.COM along your path and put 
DINDEX.BIN, DICT.BIN, AUXDICT.TXT, and FIXUP.COM all in the same 
directory.  I put mine in user 6.  Do NOT make them public files. 
Then run SETDRU first on CORRECT.COM as follows and exactly in 
this order:
             
Redirect call to DINDEX .BIN to user 6
Redirect call to DICT   .BIN to user 6
Redirect call to AUXDICT.TXT to user 6
Redirect call to FIXUP  .COM to user 6

Then run SETDRU on FIXUP.COM as follows:

Redirect call to DINDEX .BIN to user 6
Redirect call to AUXDICT.TXT to user 6
Redirect call to XXPED  .$$$ to user 6
Redirect call to AUXDICT.TXT to user 6

I've left out several details of the entire alias for which you 
should consult my CORRECT10.LBR but this takes care of the 
crucial part.
              
    CORRECT-IT can finally be used with any BDOS, with or without 
a provision for public files, and with or without an otherwise 
valuable safety feature.

    It _is_ rather ironic that in this age of modern operating 
systems with public files and safety features I would rely on for 
a solution deliberately overriding one of the modern features and 
utilize instead something from the bad old days of the CP/M 
yesteryear, but all sorts of things in life are ironic.

    I thoroughly enjoyed this adventure in debugging.  Doing this 
kind of work gives me a great deal of satisfaction.
   
    Happy CORRECT-ing...

                                       - Rick Charnes

I greatly solicit comments, suggestions and other assorted 
hoots and catcalls regarding the above flights of fancy.  I can 
be reached at Z-Node Central (Los Altos, CA), Ladera Z-Node (Los 
Angeles), Newton Centre Z-Node (Boston), Lillipute Z-Node 
(Chicago), or, God forbid, by voice at (415) 826-9448.  


                         --==**==--
